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' Abstract : In an optimal control framework, we consider the value Vr(x) of the problem 

CN ■ starting from state x with finite horizon T, as well as the value V\{x) of the A-discounted problem 

starting from x. We prove that uniform convergence (on the set of states) of the values Vr(-) 
as T tends to infinity is equivalent to uniform convergence of the values V\{-) as A tends to 0, 
and that the limits are identical. An example is also provided to show that the result does not 
hold for pointwise convergence. This work is an extension, using similar techniques, of a related 
result in a discrete-time framework 1151. 
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1 Introduction 



Finite horizon problem of optimal control have been studied intensively since the pioneer work 
of Stekhov, Pontryagin, Boltyanski, Hestenes, Bellman and Isaacs during the cold war - see for 
instance [3lll31fT^] for major references, or [8] for a short, clear introduction. A classical model 
considers the following controlled dynamic over M+ 




' 1 2/(0) = y 



where y is a function from M+ to M. n , yo is a point in M n , u is the control function which belongs 
to U, the set of Lebesgue- measurable functions from M+ to a metric space U and the function 
/ : M. n X U — > W 1 satisfies the usual conditions, that is: Lipschitz with respect to the state 
variable, continuous with respect to the control variable and bounded by a linear function of the 
state variable, for any control u. 

Together with the dynamic, an objective function g is given, interpreted as the cost function 
which is to be minimized and assumed to be Borel- measurable from W 1 x U to [0,1]. For each 
finite horizon t G]0, +oo[, the average value of the optimal control problem with horizon t is 
defined as 

1 /"* 

Vt(yo) = inf - / g(y(s,u,ya),u(s))ds (1.2) 



u£U t JQ 

It is quite natural to define, whenever the trajectories considered are infinite, for any discount 
factor A > 0, the A-discounted value of the optimal control problem, as 

p+oo 

V x (y )=mf\ e- Xs g(y(s,u,y ),u(s))ds (1.3) 

In this framework the problem was initially to know whether, for a given finite horizon T and 
a given starting point yo, a minimising control u existed, solution of the optimal control problem 
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(T, j/o) ■ Systems with large, but fixed horizons were considered and, in particular, the class of 
"ergodic" systems (that is, those in which any starting point in the state space Q is controllable 
to any point in fi) has been thoroughly studied. These systems are asymptotically independent 
of the starting point as the horizon goes to infinite. When the horizon is infinite, the literature 
on optimal control has mainly focussed on properties of given trajectories as the time tends to 
infinity. This approach corresponds to the uniform approach in a game theoretical framework 
and is often opposed to the asymptotic approach (described below), which we have considered 
in what follows, and which has received considerably less attention. 

In a game-theoretical, discrete time framework, the same kind of problem was considered 
since |18j , but with several differences in the approach: 1) the starting point may be chosen 
at random, i.e, a probability // may be given on f2, which randomly determines the point from 
which the controller will start the play; 2) the controllability-ergodicity condition is generally 
not assumed; 3) because of the inherent recursive structure of process played in discrete time, 
the problem is generally considered for all initial states and time horizons. 

For these reasons, what is called the "asymptotic approach" — the behavior of Vt(-) as 
the horizon t tends to infinity, or of V\{-) as the discount factor A tends to zero — as been 
more studied in this discrete-time setup. Moreover, when it is considered in Optimal Control, 
in most cases OH] an ergodic assumption is made which not only ensures the convergence of 
Vt(yo) to some V, but also forces the limit function V to be independent of the starting point 
HQ. The general asymptotic case, in which no ergodicity condition is assumed, has been to our 
knowledge studied for the first time recently. In [5j[I7] the authors prove in different frameworks 
the convergence of Vt(-) and V\(-) to some non-constant function V(yo). 

Some important, closely related questions are the following : does the convergence of Vj(-) 
imply the convergence of V\(-) ? Or vice versa ? If they both converge, does the limit coincide ? 
A partial answer to these questions goes back to the beginning of the 20 th century, when Hardy 
and Littlewood proved (see that for any sequence of bounded real numbers, the convergence 
of the Cesaro means is equivalent to the convergence of their Abel means, and that the limits 
are then the same : 

Theorem 1 (Hardy-Littlewood 1914) For any bounded sequence of reals {a n } n >i, define 
Vn = l Ei=i Oi and V\ = X - A)*-V Then, 

liminf V n < liminf V\ < limsup V\ < limsup V n . 

n-»+oo A^O A->0 

Moreover, if the central inequality is an equality, then all inequalities are equalities. 

Noticing that {a n } can be viewed as a sequence of costs for some deterministic (uncontrolled) 
dynamic in discrete-time, this results gives the equivalence between the convergence of Vt and the 
convergence of V\, to the same limit. In 1971, setting Vt = \ Jq g(s)ds and V\ = A Jg + °° e~ Xs g(s)ds, 
for a given Lebesgue-measurable, bounded, real function g, Feller proved in [9] that the same 
result holds for continuous-time uncontrolled dynamics. 

Theorem 2 (Feller 1971) 

liminf V n < liminf V\ < limsup V\ < limsup V n . 

Moreover, if the central inequality is an equality, then all inequalities are equalities. 

In 1992, Lehrer and Sorin |15| considered a discrete-time controlled dynamic, defined by a cor- 
respondence r : 0, =4 0, with nonempty values, and by g, a bounded real cost function defined 
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on O. A feasible play at z £ O is an infinite sequence y = {y n }n>i such that y\ = z and 
y-n+i S r(y n ). The value functions are defined by V n (z) = inf ^^7=1 9 '(Vi) ano - respectively 
V\(yo) = inf A 5^i=i (1 ~~ ^Y~ 1 9(yi)> where the infima are taken over the feasible plays at z. 

Theorem 3 (Lehrer-Sorin 1992) 

lim V n (z) = V(z) uniformly on £1 <J=^ lim V\{z) = V(z) uniformly on 

n— >+oo A— ¥0 

This result establishes the equivalence between uniform convergence of V\(yo) when A tends to 
and uniform convergence of V n (yo) as n tends to infinity, in the general case where the limit 
may depend on the starting point y$. The uniform condition is necessary: in the same article, 
the authors provide an example where only pointwise convergence holds and the limits differs. 

In 1998, Arisawa (see [2]) considered a continuous-time controlled dynamic and proved the 
equivalence between the uniform convergence of V\ and the uniform convergence of Vt in the 
specific case of limits independent of the starting point. 

Theorem 4 (Arisawa 1998) Let del, then 

lim Vt(z) = d uniformly on Q <^=^ lim V\{z) = d uniformly on f2. 

i-5>+oo A^0+ 

This does not settle the general case, in which the limit function may depend on the starting 
point0. For a continuous-time controlled dynamic in which Vt(yo) converges to some function 
V(yo), dependent on the state variable yo, as t goes to infinity, we prove the following 

Theorem 5 Vt(yo) converges to V(yo) uniformly on ft, if and only ifV\{yo) converges to V(yo) 
uniformly on Q. 

In fact, we will prove this result in a more general framework, as described in section 2. Some 
basic lemmas which occur to be important tools will also be proven on that section. Section 3 
will be devoted to the proof of our main result. Section 4 will conclude by pointing out, via 
an example, the fact that uniform convergence is a necessary requirement for the Theorem [5] 
to hold. A very simple dynamic is described, in which the pointwise limits of Vj(-) and V\(-) 
exist but differ. It should be noted that our proofs (as well as the counterexample in Section H]) 
are adaptations in this continuous-time framework of ideas employed in a discrete-time setting 
in |15] , In the appendix we also point out that an alternate proof of our theorem is obtained 
using the main theorem in |15] as well as a discrete/continuous equivalence argument. 

For completeness, let us mention briefly this other approach, mentioned above as the uni- 
form approach, and which has also been deeply studied, see for exemple 0EHTU]). In these 
models, the optimal average cost value, i.e the Vt, is not taken over a finite period of time [0, t], 
which is then studied for t growing to infinite, as in [2^l9^[Tl ] [T5 t ll7] or in our framework. On 
the contrary, only infinite trajectories are considered, among which the value Vt is defined as 
inf ue ^sup T>t ^ Jq g(y(s,u,yo),u(s))ds, or some other closely related variation. The asymptotic 
behavior, as t tends to infinity, of the function Vt has also been studied. In |10j . both A-discounted 
and average evaluations of an infinite trajectory are considered and their limits are compared. 
However, we stress out that the asymptotic behavior of those quantities is in general! not related 
to the asymptotic behavior of Vt and V\. 

'Lemma 6 and Theorem 8 in [2] deal with this general setting, but we believe them to be incorrect since they 
are stated for pointwise convergence and, consequently, are contradicted by the example in Section 2] 
2 The reader may verify that this is indeed not the case in the example of Section [4] 
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2 Model 



2.1 General framework 

We consider a deterministic dynamic programming problem in continuous time, denned by a 
measurable set of states f2, a subset T of Borel- measurable functions from R + to $7, and a 
bounded Borel-measurable real- valued function g defined on 0. Without loss of generality we 
assume g : 0, — > [0,1]. For a given state x, define T(x) := {X E 7", -X"(0) = x} the set 
of all feasible trajectories starting from x. We assume V{x) to be non empty, for all x E J). 
Furthermore, the correspondence T is closed under concatenation: given a trajectory X E 
with X(s) = y, and a trajectory Y E r(y), the concatenation of X and Y at time s is 

Xo,yJ v /W [ H^ S (2.1) 

and we assume that X o s Y E 

We are interested in the asymptotic behavior of the average and the discounted values. It is 
useful to denote the average payoff of a play (or trajectory) X E T(x) by: 

lt(X) := - f g(X(s))ds (2.2) 
1 Jo 

r+oo 

lx(X) := A / e- Xs g(X(s))ds (2.3) 
J o 

This is defined for t, A E (0,oo). Naturally, we define the values as: 

V t {x) = inf 7t (X) (2.4) 
xer(x) 

Va(x) = inf 7A (X) (2.5) 
xer(x) 



Our main contribution is the following : 



Theorem 6 (A) V\ — > V , uniformly on £1 <^=^ (B) Vt — > V , uniformly on Q. 

t->oo 

Notice that our model is a natural adaptation to the continuous-time framework of determin- 
istic dynamic programming problems played in discrete time ; as it was pointed out during the 
introduction, this theorem is an extension to the continuous-time framework of the main result 
of |15| . and our proof use similar technics. 

This result can be applied to the model presented in section [1} denote Cl = M. d x U and for 
any (yo,uo) E fi, define r(yo,^o) = u (')) \ u £M,u(0) = uq and y is the solution of (|l.ip .| 

Then Q, T and g satisfy the assumptions of this section. Defining Vt and V\ as in (|2.4p and (|2.5p 
respectively, since the solution of (jl.ip does not depend on u(0) we get that 

Vt(yo,uo) = V t (yo) 
V\(y ,u ) = V\(y ). 

Theorem [S] applied to V thus implies that Vt converges uniformly to a function V in Q if and 
only if V\ converges uniformly to V in fi. 
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2.2 Preliminary results 



We follow the ideas of |15] . and start by proving two simple lemmas yet important tools, that 
will be used in the proof. The first establishes that the value increases along the trajectories. 
Then, we prove a convexity result linking the finite horizon average payoffs and the discounted 
evaluations on any given trajectory. 

Lemma 1 Monotonicity (compare with Proposition 1 in fl5\/) 
For all X G T , for all s > 0, we have 

liminf V t (X(0)) < liminf V t (X(s)) (2.6) 

t— too t—>-oo 

liminf V x (X(0)) < liminf V x (X(s)) (2.7) 

A— >0 A— >0 

Proof. Set y := X(s) and x := X(0). For e > 0, take TeK + such that ^ < e. Let t > T 
and take a e-optimal trajectory for Vt , i.e. Y E T(y) such that 7t(V) < Vt(y) + £■ Define the 
concatenation of X and Y at time s as in (|2,ip . where X o s K is in G T(x) by our hypothesis. 
Hence 

V t+s (x)< lt+s {Xo s Y) = -f- ls (X) + ^-^(Y) 

< e + lt (Y) 

< 2e + V t (y). 

Since this is true for any t >T the result follows. 

Similarly, for the discounted case let A > be such that A Jq e~ Xor dr = 1 - e x ° s < e. Let 
A £ (0, Ao) and take Y G r(y) a e-optimal trajectory for V\{y). Then: 

ps poo 

V x {x) < 7A (X o s Y) = X e- Xr g(X(r))dr + A / e~ Xr g(Y(r - s))dr 

JO Js 

< e + e~ Xs lx {Y) 

< 2e + V x (y). 

Again, this is true for any A G (0, Ao), and the result follows. 

■ 

Lemma 2 Convexity (compare with equation (1) in f!5\j) 
For any play IeT, for any A > 0: 

oo 



7 aP0 = / 7s(X)fi x (s)ds, (2.8) 
Jo 

where fi x (s)ds := X 2 se~ Xs ds is a probability density on [0, +oo]. 



Proof. It is enough to notice that the following relation holds, by integration by parts : 

poo poo / -i ps \ 

7 aP0 = A / e- Xs g(X(s))ds = A 2 / se~ Xs - / g(X(r))dr ds, 



o 



and that f™ A 2 se" As ds = 1. ■ 
The probability measure fi x will play an important role in the rest of the paper. Denoting 

M(a, j3; A) := / /-i x (s)ds = e~ Xa (l + Xa) — e~ X ^(l + A/3), we prove here two estimates that will 

J a 

be helpful in the next section. 
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Lemma 3 The two following results hold (compare with Lemma 3 in fl5\j): 

(i) Vt > 0,3e such that Me < e , M((l - e)t,t;l/t) > e/2e. 

(ii) VS > 0, 3e such that Ve < e , Vi > 7 M(et, (1 - e)i; 1/Ve) >1-S. 
Proof. Notice that in these particular cases, M does not depend on t: 

(i) M(t(l - e),t; l/t) = (2 - e)e~ 1+t - 2e~ l = ~(e + o(e)) > ^, for e small enough. 

(ii) M(te,t(l - e); 1/y/et) = (1 + ^)e~^ - (1 - + y/eje" 1 ^^. This expression tends 
to 1 as e —7- 0, hence the result. 



3 Proof of Theorem [6] 
3.1 From V t to V x 

Assume (B) : Vt(uS) converges to some V(w) as t goes to infinity, uniformly on f2. 
Proposition 4 f Compare with Proposition 4 in }15\/) 

For all e > 0, there exists \q > such that V\(x) > V{x) — e for every x £ 0, and for all 
AG (0,A ]. 



Proof. Let T be such that \\Vt — V\\oo < e/2 for every t > T. Choose Ao > such that 

se - As ds = l - (l + AT)e" AT > 1 - e/4, for every A G (0, A ). Fix A G (0, A ) and take a 

play Y" G r(x) which is e/4-optimal play for V^(x). Since 7 S (-X") > 0, the convexity formula (|2.8p 
from Lemma [2] gives: 

^A(x) + e/4> 7A (y) > + A 2 y^ se- As 7s (y) 

/■oo 

> A 2 / se~ Xs V s (x) 

JT 

> (l-e/A)(V(x)-e/2) 

= V(x)-e/4V(x)-e/2 + e 2 /8 

> V(x) - 3/4e 



Lemma 5 ( Compare with Lemma 8 in f!5\j) 

Ve > 0,3M such that for allt > M, Vx G fi, i/iere is a play X G r(x) suc/i that ^ S (X) < V(x) + e 
for all s G [ei, (1 - e)t]. 

Proof. By (B) there exists M such that \\V r - V\\ < y for all r > eM. Given t > M and 

x G fi, let X G T(x) be a play (from x) such that 7t(-X") < Vj(x) + y. For any s < (1 — e)i, 
t — s > et> eM so Proposition Q] (Monotonicity) imply that 

2 2 

Vt- s (X(s)) > V(X(s)) -j> V{x) - j. (3.1) 
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Since V{x) + e /3 > Vt(x), we also have: 



e 2 \ /„, , e 2 



t(y(x) + 2-l > t\V t {x) + - 



> t lt (X)= f g{X(r))dr + f g(X(r))dr 

JO Js 

> s ls (X) + (t - S )14_ S (X( S )) 

> s 7s (*) + (*-«) (V(*) - y J by dan). 



Isolating 7s (AC) we get: 



7s (X) < F(x)+e 2 - 
s 

< F(z) + e, for s/e>i, 
and we have proved the result for all s G [ei, (1 — e)t]. ■ 
Proposition 6 / Compare with Lemma 9 in ]lb^) 

\/5 > 0, 3Ao such that \/x G f2, for all A G (0, Ao], we have V\{x) < V(x) + 5. 

Proof. By Lemma [3] (ii), one can choose e small enoug h such that M(ei, (1 - e)t; > 1 - § , 

for any i. In particular, we can take e < |. Using Lemma[5]with (5/2, we get that for t > to (and 
thus for Xt := ^= < ^/j) an d for anv £ G £1, there exists a play X G such that 



7At (X) < 5/2 + X t 2 f se XtS ls {X)ds 

Jet 

< 8/2 + 1- (V(x) + 8/2). 



Propositions H] and [6] establish the first part of Theorem [6j (B) =4> (A). 



3.2 From V x to V t 

Now assume (^4) : V\(lj) converges to some W(uj) as A goes to 0, uniformly on fL We start by 
a technical Lemma: 



Lemma 7 ( Compare with Proposition 2 in f!5\j) 

Let e > 0. For all x G ft and t > 0, and for any trajectory Y G T(x) which is e/2-optimal for the 
problem with horizon t, there is a time L G [0, i (1 — e/2)] such that, for all T G]0, t — L\: 

i£ + g(Y(s))ds<V t (x) + e. 

Proof. Fix Y G T(x) some e/2-optimal play for Vt(x). The function s -+ J S (Y) is continuous 
on ]0, t] and satisfies jt(Y) < Vt(x) + e/2. The bound on g implies that j r (Y) < Vt(x) + e for all 
r G [t{\-e/2),t). 

Consider now the set {s G]0,i] | J S (Y) > Vt(x) + e}. If this set is empty, then take L = 
and observe that ± j Q r g(Y(s))ds < V t (x) + e,Vr G]0,i]. 
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Otherwise, let L be the superior bound of this set. Notice that L < t(l — e/2) and that by 
continuity jl(Y) = Vt(x) + e. Now, for any T £ [0, t — L], 



V t (x)+e > jl+t(Y) 



L T / 1 



L+T 



L+T 



L + T ' L + T \T 
L , T / 1 



L + T 

and the result follows. 



{V t {x) + e) + 1 ^{^f L + g(Y(s))ds 



Proposition 8 f Compare with Lemma 6 in ]lb^) 

Ve > 0, 3T such that for all t > T we have Vt(x) > W(x) — e, for all x £ Q. 

/•oo 

Proof. Let A be such that \\V\ - W\\ < e/8, and T such that A 2 / se~ Xs ds < e/8. Proceed 

JTe/i 

by contradiction and suppose that e > is such that for every T, there exists to >T and a state 
xq £ f] such that Vt (xo) < W(xq) — e. 

Using Lemma [7] with e/2, we get a play Y £ r(xo) and a time L £ [0,£o(l — such that, 
Vs £ [0, to — L] (and, in particular, Vs £ [0, toe/4\): 



1 



HS ^ (^(r)) < V t0 (x ) + e/2 < W(x ) - e/2. 



Thus, 



W(F(L))-e/8 < V X (Y(L)) 

poo 

< A / e" As 5 (y(L + s))ds 
Jo 



toe/4 /i rL+s 



A 2 / se" As 



o 



j + g{Y{r))dr^j ds + e/8 



< W(x )-e/2 + e, 
= W(x ) - 3e/8. 



This gives us W(Y(L)) < W{xq) — e/4, contradicting Proposition [T] (Monotonicity). ■ 
Proposition 9 ( Compare with Lemma 7 in flty) 

Ve > 0, 3T such that for all t >T we have Vt(x) < W{x) + e, for all x £ £1. 

Proof. Otherwise, 3e > such that VT, 3t > T and x £ Q with V t {x) > W{x) + e. For any 

1 /" s 

X £ r(x) consider the (continuous in s) payoff function 7 S (X) = — g{X(r))dr . Of course, 

s Jo 

lt(X) > Vj(x) > W(a;) + e. Furthermore, because of the bound on g, 

lr{X) > W(x) + e/2, Vr £ [t{l - e/2),t]. 

By Lemma[3l we can take e small enough, so that for all t, M(t(l — e/2), t; 1/t) > e/4e holds. 
We set 5 := e/4e. 

By Proposition [HJ there is a K such that Vt > W{x) — 5e/8, for all t > K. 
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For K fixed, we consider M(0, K; l/t) = 1 - e" A '/*(l + K/t) function of t. Clearly, it 
tends to as t tends to infinity, so let t be such that this quantity is smaller than <5e/16. Also, 
let t be big enough so that \\V\u — W\\ < Se/5, which is a consequence of assumption (A). 

On the following, we set A := l/t and consider the A-payofF of some play X £ We'll 
split the integral over [0, +00] in three parts : 1C = [0, K],1Z = [t(l — e/2),t], and (/C U 1Z) C . 
The three parts are clearly disjoint since t » K. We have seen that /j,\(s)ds = X 2 se~ Xs ds is 
a probability measure on [0, +00], for any A > 0. Then by the Convexity formula (|2.8p . we can 
write: 



Tx (X) = / ls (X) H (ds) + / T s(X) H (ds) + / ls(X) H (ds) . 
\Jk Jti J(icuii) c I 



Recall that 



ls{X\ K > 
Js(X) l{Kun) c > W(x)-Se/8 
ls(X) ]n > W(x) + e/2 



It is straightforward that 



T X (X) > + 8 ■ (W{x) + e/2) + (1 - S - 6e/16) ■ (W(x) - Se/8) 

= W(x) + 5e[ 1 

w V2 16 8 8 64 

> W{x) + 5e/4. 

This is true for any play, so its infimum also satisfies V^(x) > W(x) + 5e/4, which is a contra- 
diction, for we assumed that < W(x) + 5e/5. ■ 

Propositions [8] and [9] establish the second half of Theorem [6]: (A) =^ (B). 



4 A counter example for pointwise convergence 

In this section we give an example of an optimal control problem in which both Vt(-) and V\(-) 
converge pointwise on the state space, but to two different limits. As implied by Theorem [6] the 
convergence is not uniform on the state space. 

Lehrer and Sorin were the first to construct such an example |15| . in the discrete-time frame- 
work. We consider here one of its adaptation in continuous time, which was studied as Example 

5 in [TTIpl where the notations are the same that in Section [TJ 

— The state space is S7 = 

— The payoff function is given by g(x, y) = if x G [1,2], 1 otherwise. 

— The set of control is U = [0, 1]. 

— The dynamic is given by f(x,y,u) = (y,u) (thus £1 is forward invariant.) 

3 We thank Marc Quincampoix for pointing out this example to us, which is simpler that our original one. 
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An interpretation is that the couple (x(t),y(t)) represents the position and the speed of some 
mobile moving along an axis, and whose acceleration u(t) is controlled. Observe that since 
U = [0, 1], the speed y(t) increases during a play. We claim that for any (xo, yo) £ K+, Vt{xQ, yo) 
(resp V\(xo,yo)) converges to V(xo,yo) as t goes to infinity (resp. converges to W(xo,yo) as A 
tends to 0, where: 



V(x ,y ) 



W(x ,y ) 




(2-x )' 2 - x 

\ and W(0, 0) 



if yo > or xq > 2 
if yo = and 1 < xq < 2 
if yo = and xq < 1 

if yo > or xo > 2 
if yo = and 1 < xo < 2 
if yo = and xq < 1 



| ; the proof for yo 



and < xq < 1 is 



Here we only prove that V(0, 0) 
similar and the other cases are easy. 

First of all we prove that for any t or A and any admissible trajectory (that is, any function 
X(t) = (x(t),y(t)) compatible with a control u(t)), starting from (0,0), 7t(A") > \ and 7a(-^) > 
|. This is clear if x(l) is identically 0, so consider this is not the case. Since the speed y(t) is 
increasing, we can define t\ and t2 as the time at which x(ti) 
and moreover we have t2 < 2ti. Then, 



1 and x{t2) = 2 respectively, 



lt(X) 




min(t,ti) ft 

ds + ds 

J min(t,t2) 



1 + min ( 1 



min 1 



> 1 + min 



t2 
1,— 

' 2t 



mm 



t2 



> - 



and 



7a(*) 



> 
> 

> 



Xe~ Xs ds + 



Xe~ Xs ds 



t-2 



minjl 

a>0 

3 
4" 



\i 2 

2-1 



a + a 2 } 



On the other hand, one can prove |17| that limsupVt(0, 0) < \ : in the problem with hori- 
zon t, consider the control "u(s) = 1 until s = | and then 0". Similarly one proves that 
limsupVx(0,0) < |: in the A-discounted problem, consider the control "u(s) = 1 until s = 
and then 0". 

So the functions Vt and V\ converge pointwise on fi, but their limits V and W are different, 
since we have just shown V(0, 0) ^ W(0, 0). One can verify that neither convergence is uniform 
on Q by considering Vt(l,e) and V\(l,e) for small positive e. 
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Remark 1 One may object that this example is not very regular since the payoff g is not con- 
tinuous and the state space is not compact. However a related, smoother example can easily be 
constructed: 

1. The set of controls is still [0, 1]. 

2. The continuous cost g(x) is equal to 1 outside the segment [0.9,2.1], to on [1,2], and 
linear on the two remainings intervals. 

3. The compact state space is Q = {(x,y)\0 <y< \/2x < 2\^2}. 

4- The dynamic is the same that in the original example for x 6 [0,3], and f(x,y,u) = 
((4 — x)y, (4 — x)u) for 3 < x < 4. The inequality y{t)y'(t) < x'(t) is thus satisfied on any 
trajectory, which implies that Q is forward invariant under this dynamic. 

With these changes the values Vj(-) and V\(-) still both converge pointwise on £1 to some V(-) 
and W{-) respectively, and U(0,0) / W(0,0). 

5 Possible extensions 

— We considered the finite horizon problem and the discounted one, but it should be possible 
to establish similar Tauberian theorems for other, more complex, evaluations of the payoff. 
This was settled in the discrete time case in |16| . 

— It would be very fruitful to establish necessary or sufficient conditions for uniform con- 
vergence to hold. In this direction we mention |17| in which sufficient conditions for the 
stronger notion of Uniform Value (meaning that there are controls that are nearly optimal 
no matter the horizon, provided it is large enough) are given in a general setting. 

— In the discrete case an example is constructed in |16] in which there is no uniform value 
despite uniform convergence of the families Vt and V\. It would be of interest to construct 
such an example in continuous time, in particular in the framework of section [TJ 

— It would be very interesting to study Tauberian theorems for dynamic systems that are 
controlled by two conflicting controllers. In the framework of differential games this has 
been done recently (Theorem 2.1 in pQ): an extension of Theorem 2] has been accomplished 
for two player games in which the limit of Vt or V\ is assumed to be independent of 
the starting point. The similar result in the discrete time framework is a consequence of 
Theorems 1.1 and 3.5 in |12j . Existence of Tauberian theorems in the general setup of two- 
persons zero-sum games with no ergodicity condition remains open in both the discrete 
and the continuous settings. 
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Appendix 

We give here another proo^l of Theorem by using the analoguous result in discrete time |15| 
as well as an argument of equivalence between discrete and continuous dynamic. 

Consider a deterministic dynamic programming problem in continuous time as defined in 
section 12, 1| with a state space f2, a payoff g and a dynamic V. Recall that, for any oj £ Vt, 
V{oj) is the non empty set of feasible trajectories, starting from oj. We construct an associated 
deterministic dynamic programming problem in discrete time as follows. 

Let H = (J x [0,1] be the new state space and let g be the new cost function, given by 
g(oj,x) = x. We define a multivalued- function with nonempty values V : £1 z4 by 

(oj, x) e T(oj', x') <^ 3X e r(w'), with X(l) = oj and f g{X(t))dt = x. 

Jo 

Following [15] . we define, for any initial state oj = (oj,x) 

1 " 

v n (u) = inf-V^w;) (5.1) 

i=l 
+oo 

v x (u) = infA^(l-Ar 1 g(5 i ) (5.2) 
i=i 

where the infima are taken over the set of sequences {cjj}j 6 N such that ujq = oj and G T(oJi) 
for every i > 0. 

Theorem [6] is then the consequence of the following three facts. Firstly, the main theorem 
of Lehrer and Sorin in |15| . which states that uniform convergence (on f2) of v n to some v is 
equivalent to uniform convergence of v\ to the same v. 

Secondly, the concatenation hypothesis (|2.ip on V implies that for any (oj,x) S 

v n (oj,x) = V n (oj) 

where Vt(oj) = inf^er^) \ Jq g(X(s))ds, as defined in equation (|2.4p . Consequently, because of 
the bound on g, for any t £ M+ we have 

2 

\V t (oj) - v [t] (oj,x)\ < — 

where [t\ stands for the integral part of t. 

Finally, again because of hypothesis (|2.ip . for any A G]0, 1], 

r+oo 

v x (oj,x)= inf A/ (1 - X) lti g{X{t))dt. 
xer(u) J 

Hence, by equation (|2.5p and the bound on the cost function, for any A £]0, 1], 



\V x (oj)-v x (oj,x)\<\ / (l-A)W -e~ xt 



+oo 







dt 



which tends uniformly (with respect to x and oj) to as A goes to by virtue of the following 
lemma. 



4 We thank Frederic Bonnans for the idea of this proof 
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Lemma 10 The function 

A — > X 

converges to as X tends to 



o 



(1_A)L*J -e" At 



Proof. Since A J + °°(l — A)W = A Jq + °° e A *(ii = 1, for any A > 0, the lemma is equivalent to 
the convergence to of 







E(X) := A / (l-A)W -e~ xt 



dt 



where [x]+ denotes the positive part of x. Now, from the relation 1 — A < e A , true for any A, 
one can easily deduce that, for any A > 0, t > 0, the relation (1 — A) L*J e A * < e A holds. Hence, 



E(X) = X / e~ xt (1-A) L V*-1 
Jo L 

r+oo 

< X / e- At (e A - l)dt 
Jo 



= e x -l 

which converges to as A tends to 0. ■ 
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